Multi-Model Switching On a Collision Mitigation System

ABSTRACT

Systems and methods for controlling an autonomous vehicle are provided. In one example embodiment, a computer-implemented method includes receiving data indicative of an operating mode of the vehicle, wherein the vehicle is configured to operate in a plurality of operating modes. The method includes determining one or more response characteristics of the vehicle based at least in part on the operating mode of the vehicle, each response characteristic indicating how the vehicle responds to a potential collision. The method includes controlling the vehicle based at least in part on the one or more response characteristics.

FIELD

The present disclosure relates generally to operating modes of anautonomous vehicle.

BACKGROUND

An autonomous vehicle is a vehicle that is capable of sensing itsenvironment and navigating without human input. In particular, anautonomous vehicle can observe its surrounding environment using avariety of sensors and can attempt to comprehend the environment byperforming various processing techniques on data collected by thesensors. Given knowledge of its surrounding environment, the autonomousvehicle can identify an appropriate motion path through such surroundingenvironment.

SUMMARY

Aspects and advantages of embodiments of the present disclosure will beset forth in part in the following description, or may be learned fromthe description, or may be learned through practice of the embodiments.

One example aspect of the present disclosure is directed to acomputer-implemented method for controlling an autonomous vehicle. Themethod includes receiving, by a computing system comprising one or morecomputing devices, data indicative of an operating mode of theautonomous vehicle, the autonomous vehicle configured to operate in aplurality of operating modes. The method includes determining, by thecomputing system, one or more response characteristics of the autonomousvehicle based at least in part on the operating mode of the autonomousvehicle, each response characteristic indicating how the autonomousvehicle responds to a potential collision. The method includescontrolling, by the computing system, the autonomous vehicle based atleast in part on the one or more response characteristics.

Another example aspect of the present disclosure is directed to acomputing system for controlling an autonomous vehicle. The computingsystem includes one or more processors, and one or more tangible,non-transitory, computer readable media that collectively storeinstructions that when executed by the one or more processors cause thecomputing system to perform operations. The operations include receivingdata indicative of an operating mode of the autonomous vehicle, whereinthe autonomous vehicle is configured to operate in a plurality ofoperating modes. The operations include determining one or more responsecharacteristics of the autonomous vehicle based at least in part on theoperating mode of the autonomous vehicle, each response characteristicindicating how the autonomous vehicle responds to a potential collision.The operations include controlling the autonomous vehicle based at leastin part on the one or more response characteristics.

Yet another example aspect of the present disclosure is directed to anautonomous vehicle. The autonomous vehicle includes one or moreprocessors, and one or more tangible, non-transitory, computer readablemedia that collectively store instructions that when executed by the oneor more processors cause the autonomous vehicle to perform operations.The operations include receiving data indicative of an operating mode ofthe autonomous vehicle, wherein the autonomous vehicle is configured tooperate in a plurality of operating modes. The operations includedetermining one or more response characteristics of the autonomousvehicle based at least in part on the operating mode of the autonomousvehicle, each response characteristic indicating how the autonomousvehicle responds to a potential collision. The operations includecontrolling the autonomous vehicle based at least in part on the one ormore response characteristics.

Other example aspects of the present disclosure are directed to systems,methods, vehicles, apparatuses, tangible, non-transitorycomputer-readable media, and memory devices for controlling anautonomous vehicle.

These and other features, aspects, and advantages of various embodimentswill become better understood with reference to the followingdescription and appended claims. The accompanying drawings, which areincorporated in and constitute a part of this specification, illustrateembodiments of the present disclosure and, together with thedescription, serve to explain the related principles.

BRIEF DESCRIPTION OF THE DRAWINGS

Detailed discussion of embodiments directed to one of ordinary skill inthe art are set forth in the specification, which make reference to theappended figures, in which:

FIG. 1 depicts an example system overview according to exampleembodiments of the present disclosure;

FIG. 2 depicts an example vehicle control system for selecting anoperating mode and controlling an autonomous vehicle to avoid apotential collision based at least in part on the operating modeaccording to example embodiments of the present disclosure;

FIG. 3 depicts a configurable data structure for storing one or moreoperating mode(s) according to example embodiments of the presentdisclosure;

FIGS. 4A and 4B depict state diagrams of a collision mitigation systemaccording to example embodiments of the present disclosure;

FIG. 5 depicts a flow diagram of switching an operating mode andcontrolling an autonomous vehicle based at least in part on theoperating mode according to example embodiments of the presentdisclosure; and

FIG. 6 depicts example system components according to exampleembodiments of the present disclosure.

DETAILED DESCRIPTION

Reference now will be made in detail to embodiments, one or moreexamples of which are illustrated in the drawings. Each example isprovided by way of explanation of the embodiments, not limitation of thepresent disclosure. In fact, it will be apparent to those skilled in theart that various modifications and variations can be made to theembodiments without departing from the scope or spirit of the presentdisclosure. For instance, features illustrated or described as part ofone embodiment can be used with another embodiment to yield a stillfurther embodiment. Thus, it is intended that aspects of the presentdisclosure cover such modifications and variations.

Example aspects of the present disclosure are directed to switchingbetween a plurality of operating modes on a collision mitigation systemlocated on-board an autonomous vehicle to improve the safety,customizability, and flexibility of the autonomous vehicle. A collisionmitigation system is a safety feature meant to safeguard againstfailures, shortcomings, or unforeseeable events when operating anautonomous vehicle. A collision mitigation system can detect potentialcollisions with objects in a surrounding environment of an autonomousvehicle and control the autonomous vehicle to avoid the potentialcollision. An autonomous vehicle can calibrate a collision mitigationsystem based at least in part on an operating mode of the autonomousvehicle. For example, if an autonomous vehicle is operating inautonomous mode, it can calibrate an on-board collision mitigationsystem to provide an indication to an on-board autonomy computing systemfor each potential collision that is detected. Whereas, if an autonomousvehicle is operating in manual mode, it can calibrate a collisionmitigation system to flash a warning light to a driver if a potentialcollision is substantially likely to occur (e.g., if the potentialcollision is associated with a probability of occurring that is greaterthan a predetermined value, or if a severity level of the potentialcollision is above a predetermined threshold, etc.). Conditions andtolerances of how a collision mitigation system responds to a potentialcollision are herein referred to as response characteristic(s) of theautonomous vehicle. An autonomous vehicle can receive data indicative ofan operating mode, determine one or more response characteristic(s) ofthe autonomous vehicle associated with the operating mode, and controlthe autonomous vehicle based at least in part on the one or moreresponse characteristic(s). In this way, the autonomous vehicle caninclude a multi-modal collision mitigation system that improves thesafety of the vehicle and objects surrounding the vehicle.

An autonomous vehicle can include a vehicle computing system thatimplements a variety of systems on-board the autonomous vehicle (e.g.,located on or within the autonomous vehicle). For instance, the vehiclecomputing system can include an autonomy computing system (e.g., forplanning autonomous navigation), a mode manager (e.g., for setting anoperating mode of an autonomous vehicle), a human-machine interfacesystem (e.g., for receiving and/or providing information to an operatorof an autonomous vehicle) that includes one or more externalindicator(s) (e.g., for indicating a state of an autonomous vehicle), acollision mitigation system (e.g., for detecting and mitigatingpotential collisions), and a vehicle interface system (e.g., forcontrolling one or more vehicle control components responsible forbraking, steering, powertrain, etc.). The vehicle interface system ofthe autonomous vehicle can include one or more vehicle control system(s)for controlling the autonomous vehicle. For instance, the vehicleinterface system can include a braking control component, steeringcontrol component, acceleration control component, powertrain controlcomponent, etc. The vehicle interface system can receive one or morevehicle control signal(s) from one or more system(s) on-board theautonomous vehicle. The vehicle interface system can instruct the one ormore vehicle control system(s) to control the autonomous vehicle basedon the one or more vehicle control signal(s), for example, in the mannerdescribed herein to implement multi-modal switching on a collisionmitigation system.

An autonomy computing system of an autonomous vehicle can include one ormore sub-systems for planning and executing autonomous navigation. Forinstance, an autonomy computing system can include, among othersub-systems, a perception system, a prediction system, and a motionplanning system that cooperate to perceive a surrounding environment ofan autonomous vehicle and determine a motion plan for controlling themotion of the autonomous vehicle accordingly.

A mode manager of an autonomous vehicle can include one or moresub-systems for setting an operating mode of the autonomous vehicle. Forinstance, a mode manager can receive data indicative of a requestedoperating mode of an autonomous vehicle from one or more system(s)on-board the autonomous vehicle, one or more operator(s) (e.g., adriver, passenger, service technician, etc.) of the autonomous vehicle,or one or more remote computing device(s) (e.g., of an entity associatedwith the autonomous vehicle). If a requested operating mode is differentfrom a current operating mode of the autonomous vehicle, a mode managercan determine one or more prerequisite(s) to switching the operatingmode of the autonomous vehicle to the requested operating mode. A modemanager can determine whether the prerequisite(s) are met, and if so themode manager can send one or more control signal(s) to one or moresystem(s) on-board the autonomous vehicle to set the operating mode ofthe autonomous vehicle to the requested operating mode. By way ofexample, if an autonomous vehicle is operating in autonomous mode andreceives a request to switch to manual mode, then a mode manager candetermine that a prerequisite to switching the operating mode is anavailability of a driver. The mode manager can control the autonomousvehicle to display a prompt asking an operator of the autonomous vehicleto affirm that operator will take control of the autonomous vehicle todetermine if this prerequisite is met. Continuing this example, the modemanager can determine another perquisite to switching the operating modeis that the driver must be licensed to drive the autonomous vehicle. Themode manager can control the autonomous vehicle to collect and verifythe driver's license information to determine if this prerequisite ismet. The mode manager can determine yet another prerequisite toswitching the operating mode is that the autonomous vehicle is notcurrently executing a turning maneuver. The mode manager can communicatewith one or more other system(s) on-board the autonomous vehicle todetermine if this prerequisite is met. If the autonomous vehicle iscurrently executing a turning maneuver, the mode manager can wait forthe turning maneuver to be completed, or the mode manager can determinethat the prerequisite is not met.

A human-machine interface system of an autonomous vehicle can includeone or more indicator(s) for indicating a state of the autonomousvehicle. For instance, a human-machine interface system can includeexternal indicator(s) such as visual indicator(s) (e.g., a warninglight, display, etc.), tactile indicator(s) (e.g., touch responsevibration or ultrasound, etc.), and/or audible indicator(s) (e.g., aspeaker, etc.). The human-machine interface system can also include oneor more indicator(s) for receiving input from an operator (e.g., amicrophone, touch-screen display, physical buttons, levers, etc.). Ahuman-machine interface system can receive control signals from one ormore system(s) on-board an autonomous vehicle to control theindicator(s). For example, if a trajectory of an autonomous vehicleincludes an upcoming left turn, then an autonomy computing system of theautonomous vehicle can control the human-machine interface system (e.g.,via one or more control signal(s)) to activate a left turn signal lightof the autonomous vehicle. In another example, an operator of anautonomous vehicle can control the human-machine interface system toactivate a left turn signal light (e.g., via a turn signal lever).

A collision mitigation system of an autonomous vehicle can include oneor more sub-system(s) for detecting and avoiding potential collision(s).For instance, a collision mitigation system can monitor a surroundingenvironment of an autonomous vehicle using one or more sensors (e.g., aRadio Detection and Ranging (RADAR) system, one or more cameras, and/orother types of image capture devices and/or sensors) to detect apotential collision between the autonomous vehicle and an object in thesurrounding environment. When a potential collision is detected, acollision mitigation system can control an autonomous vehicle to avoidthe potential collision, based at least in part on an operating mode ofthe autonomous vehicle. For example, if an autonomous vehicle isoperating in autonomous mode and a collision mitigation system detects apotential collision, the collision mitigation system can provideinformation on the potential collision to an autonomy computing systemthat can adjust a trajectory of the autonomous vehicle to avoid thepotential collision. As another example, if an autonomous vehicle isoperating in manual mode and a collision mitigation system detects apotential collision, the collision mitigation system can control ahuman-machine interface system to display a warning light to operator(s)of the autonomous vehicle.

In some implementations, a collision mitigation system can control amotion of an autonomous vehicle to avoid a potential collision. Forexample, in the event that a potential collision persists for a timeafter a collision mitigation system provides information on thepotential collision to an autonomy computing system, the collisionmitigation system can send one or more control signal(s) to control anautonomous vehicle to execute a braking maneuver. As another example, inthe event that a potential collision persists for a time after acollision mitigation system controls an autonomous vehicle to display awarning light, the collision mitigation system can provide one or morecontrol signal(s) to control the autonomous vehicle to execute a brakingmaneuver.

A collision mitigation system can include a configurable data structurethat stores one or more operating mode(s) of an autonomous vehicle, oneor more response characteristic(s) of the autonomous vehicle, and one ormore corresponding values. The operating mode(s) can include one or morepredetermined operating mode(s). For example, the operating mode(s) caninclude a fully autonomous mode, semi-autonomous mode, manual mode,service mode, and/or other types of modes. The responsecharacteristic(s) can include at least one of a longitudinal controlresponse characteristic, lateral control response characteristic,internal indicator response characteristic, external indicator responsecharacteristic, or other characteristics.

A longitudinal control response characteristic can include, for example,a minimum distance to keep with other vehicles, a maximumacceleration/deceleration rate, a delay before automatically applyingbrakes, etc. A lateral control response characteristic can include, forexample, a minimum/maximum turning radius, a minimum distance ofadjacent cars for a lane change, etc. An internal indicator responsecharacteristic can indicate, for example, the system(s) of theautonomous vehicle to which the collision mitigation system shouldprovide information on a potential collision, a delay beforeautomatically applying brakes, etc. An external indicator responsecharacteristic can include, for example, a delay before flashing awarning light to a driver, a volume level of audible alerts, etc.

Each operating mode can be associated with one or more responsecharacteristic(s) and a corresponding value. In some implementations,each operating mode can be associated with different responsecharacteristic(s) among the one or more response characteristic(s). Forexample, an autonomous mode can be associated with responsecharacteristic(s) that indicate a collision mitigation system shouldreport to an autonomy computing system on-board the autonomous vehicleimmediately after detecting a potential collision. In another example, amanual mode can be associated with response characteristic(s) thatindicate a collision mitigation system should report to an externalindicator system on-board an autonomous vehicle, wait 500 millisecondsbefore warning a driver of a potential collision, and wait 1000milliseconds before automatically applying brakes. In someimplementations, an operating mode (e.g., service mode) can beassociated with response characteristic(s) that indicate a collisionmitigation system should report to one or more system(s) on-board anautonomous vehicle, such as both an autonomy computing system andexternal indicator system, depending on a type of service beingperformed on the autonomous vehicle.

In some implementations, a collision mitigation system can include anautonomous operating mode. The autonomous mode can be associated withone or more response characteristics and corresponding values that lowera sensitivity threshold of the collision mitigation system for detectinga potential collision. By lowering a sensitivity threshold, a collisionmitigation system can identify a greater number of potential collisions.In some implementations, a collision mitigation system can obtaininformation on a trajectory of an autonomous vehicle from one or moresystem(s) on-board the autonomous vehicle. For example, the collisionmitigation system can obtain trajectory information included in a motionplan determined by an autonomy computing system of the autonomousvehicle (e.g., when the autonomous vehicle is in a fully autonomousoperating mode). The trajectory information can include one or morefuture trajectories of the autonomous vehicle. A collision mitigationsystem can use the trajectory information to configure one or moreregions of interest in a surrounding environment of the autonomousvehicle for detecting a potential collision. For example, if atrajectory of an autonomous vehicle is to “continue forward 500 meters,then turn right,” a collision mitigation system can determine aforward-right region (with respect to the autonomous vehicle) as aregion of interest. The collision mitigation system can identify one ormore object(s) in the forward-right region and detect one or morepotential collision(s) with the object(s) in the forward-right region.By configuring the one or more regions of interest, a collisionmitigation system can reduce a number of false positives.

In some implementations, the collision mitigation system may not obtaininformation on a trajectory of an autonomous vehicle from one or moresystem(s) on-board the autonomous vehicle. For example, when theautonomous vehicle is operating in a manual operating mode, the autonomycomputing system can be prevented from generating a motion plan. In thiscase, the collision mitigation system may not obtain the trajectory ofthe autonomous vehicle. As another example, when the autonomous vehicleis operating in a manual operating mode, the vehicle's autonomycomputing system may continue to generate a motion plan includingtrajectory information. However, the autonomy system can be blocked fromcontrolling the autonomous vehicle (e.g., blocked from sending motionplans to a mobility controller, via disablement of the mobilitycontroller in the vehicle interface module, etc.). In such a case, theautonomy computing system may not provide the trajectory information tothe collision mitigation system. In some implementations, the collisionmitigation system may obtain information on a trajectory of anautonomous vehicle from one or more system(s) on-board the autonomousvehicle while in the manual mode, but may ignore such information whenperforming its operations and functions described herein.

In some implementations, a collision mitigation system can include oneor more user-configurable operating mode(s). For example, an autonomousvehicle operating in manual mode can include a plurality of driver modescorresponding to a plurality of drivers. Each driver mode can beassociated with one or more response characteristic(s) and acorresponding value to calibrate a collision mitigation system for aspecific, individual driver or type of driver. The responsecharacteristic(s) and corresponding value associated with each drivermode can indicate one or more preference(s) of a corresponding driver.For example, a first driver can set a driver mode to calibrate acollision mitigation system to warn the driver of a potential collision.In contrast, a second driver can set a driver mode to calibrate acollision mitigation system to delay or disable a warning indication.

In some implementations, a collision mitigation system can include aplurality of passenger modes corresponding to one or more passenger(s).Each passenger mode can be associated with one or more responsecharacteristic(s) and a corresponding value to calibrate a collisionmitigation system for the passenger(s). The response characteristic(s)and corresponding value associated with each passenger mode can indicateone or more preference(s) of corresponding passenger(s). For example, ifan autonomous vehicle is carrying a passenger that prefers a smoother,lower speed ride, the autonomous vehicle can set a passenger mode tocalibrate a collision mitigation system to automatically implement abraking action sooner and over a longer duration. As another example, ifan autonomous vehicle is carrying a passenger that prefers anenvironmentally friendly ride, the autonomous vehicle can set apassenger mode to calibrate a collision mitigation system to reduceenergy consumption by braking less often.

In some implementations, a collision mitigation system can include aplurality of service-type modes (e.g., for when the vehicle is operatingin a service mode) to calibrate the collision mitigation system based atleast in part on one or more preference(s) indicated by a service-typemode. For example, a service-type mode for testing an autonomy computingsystem of an autonomous vehicle can calibrate a collision mitigationsystem to report a potential collision to the autonomy computing systemand to not automatically control the autonomous vehicle to execute abraking maneuver. In another example, a service-type mode for testing ahuman-machine interface system of an autonomous vehicle can calibrate acollision mitigation system to indicate a potential collision by cyclingthrough each indicator on-board the autonomous vehicle.

The systems and methods described herein can provide a number oftechnical effects and benefits. For instance, systems and methods forswitching between a plurality of operating modes on a collisionmitigation system on-board an autonomous vehicle can have a technicaleffect of improving efficiency and flexibility in implementation ofsupplemental vehicle safety technology. An autonomous vehicle can besubject to a multitude of operating scenarios (e.g., an operatingscenario for each driver, passenger, and service-type). By calibrating acollision mitigation system of an autonomous vehicle to adjust how thecollision mitigation system responds to a potential collision, theautonomous vehicle can be optimally configured for each operatingscenario. Additionally, by storing a plurality of operating modes andrelated information in a configurable data structure, operating modescan be easily added, modified, or removed as some operating scenariosbecome obsolete and new operating scenarios become apparent.

Systems and methods for switching among a plurality of operating modeson a collision mitigation system on-board an autonomous vehicle can alsohave a technical effect of reducing false positives and false negatives.When an autonomous vehicle is operating in autonomous mode, a collisionmitigation system can inform an autonomy computing system of theautonomous vehicle immediately upon detecting each potential collision.This can generate false positives since some potential collisions canhave a higher chance of occurring than other potential collisions.However, if the collision mitigation system does not inform the autonomycomputing system of each potential collision that it detects, this cangenerate false negatives. Similarly, when an autonomous vehicle isoperating in manual mode, a driver can react adversely to a high volumeof false positives or false negatives. By enabling an autonomous vehicleto calibrate a collision mitigation system in accordance with anoperating mode of the autonomous vehicle, different responsecharacteristics can be set for autonomous mode and manual mode operationof the autonomous vehicle to reduce false positives and false negativesin each operating mode.

The systems and methods of the present disclosure also provide animprovement to vehicle computing technology, such as autonomous vehiclecomputing technology. For instance, the systems and methods hereinenable the vehicle technology to adjust response characteristics of anautonomous vehicle according to an operating mode of the autonomousvehicle. For example, the systems and methods can allow one or morecomputing device(s) (e.g., of a collision mitigation system) on-board anautonomous vehicle to detect and avoid a potential collision based atleast in part on an operating mode of the autonomous vehicle. Asdescribed herein, the autonomous vehicle can be configured tocommunicate data indicative of a potential collision and one or morecontrol signals to avoid the potential collision of a collisionmitigation system on-board the autonomous vehicle to one or more othersystem(s) on-board the autonomous system. The computing device(s) candetermine one or more response characteristic(s) associated with anoperating mode of the autonomous vehicle and control the autonomousvehicle based at least in part on the one or more responsecharacteristic(s). This can allow the autonomous vehicle to moreeffectively inform the one or more system(s) on-board the autonomousvehicle to avoid the potential collision.

Moreover, the computing device(s) can be included in a collisionmitigation system that is separate and apart from the other systemson-board the autonomous vehicle (e.g., autonomy computing system,vehicle interface system). As such, the collision mitigation system caninclude a simplified hardware architecture that is easier to upgrade,implement mode/redundancy checks, etc. This can also allow the computingdevice(s) to focus its computational resources on detecting andmitigating potential collisions, rather than allocating its resources toperform other vehicle functions (e.g., autonomous motion planning,motion plan implementation). Such use of resources can allow thecomputing device(s) to provide a more efficient, reliable, and accurateresponse to an event. Additionally, the other systems on-board theautonomous vehicle can focus on their core functions, rather thanallocating resources to the functions of the collision mitigationsystem. Thus, the systems and methods of the present disclosure can savethe computational resources of these other vehicle systems, whileincreasing performance of the collision mitigation system.

With reference now to the FIGS., example embodiments of the presentdisclosure will be discussed in further detail. FIG. 1 depicts anexample system 100 according to example embodiments of the presentdisclosure. The system 100 can include a vehicle computing system 102associated with a vehicle 103. In some implementations, the system 100can include an operations computing system 104 that is remote from thevehicle 103.

The vehicle 103 incorporating the vehicle computing system 102 can be aground-based autonomous vehicle (e.g., car, truck, bus), an air-basedautonomous vehicle (e.g., airplane, drone, helicopter, or otheraircraft), or other types of vehicles (e.g., watercraft). The vehicle103 can be an autonomous vehicle that can drive, navigate, operate, etc.with minimal and/or no interaction from a human driver. For instance,the vehicle 103 can be configured to operate in a plurality of operatingmodes 106A-C. The vehicle 103 can be configured to operate in a fullyautonomous (e.g., self-driving) operating mode 106A in which the vehicle103 can drive and navigate with no input from a user present in thevehicle 103. The vehicle 103 can be configured to operate in asemi-autonomous operating mode 106B in which the vehicle 103 can operatewith some input from a user present in the vehicle. In someimplementations, the vehicle 103 can enter into a manual operating mode106C in which the vehicle 103 is fully controllable by a user (e.g.,human driver) and can be prohibited from performing autonomousnavigation (e.g., autonomous driving). In some implementations, thevehicle 103 can be configured to operate in one or more additionaloperating mode(s). The additional operating mode(s) can include, forexample, operating mode(s) indicative of one or more preferences of adriver, passenger, or other operator(s) of the vehicle 103, operatingmode(s) indicative of a service mode, etc.

The operating mode of the vehicle 103 can be adjusted in a variety ofmanners. In some implementations, the operating mode of the vehicle 103can be selected remotely, off board the vehicle 103. For example, anentity associated with the vehicle 103 (e.g., a service provider) canutilize an operations computing system 104 to manage the vehicle 103(and/or an associated fleet). The operations computing system 104 cansend a communication to the vehicle 103 instructing the vehicle 103 toenter into, exit from, maintain, etc. an operating mode. By way ofexample, the operations computing system 104 can send a communication tothe vehicle 103 instructing the vehicle 103 to enter into the fullyautonomous operating mode 106A when providing a transportation (e.g.,rideshare) service to a user. In some implementations, the operatingmode of the vehicle 103 can be set onboard and/or near the vehicle 103.For example, the operating mode of the vehicle 103 can be selected via asecure interface (e.g., physical switch interface, graphical userinterface) onboard the vehicle 103 and/or associated with a computingdevice proximate to the vehicle 103 (e.g., a tablet operated byauthorized personnel located near the vehicle 103).

The vehicle computing system 102 can include one or more computingdevices located onboard the vehicle 103 (e.g., located on and/or withinthe vehicle 103). The computing device(s) can include various componentsfor performing various operations and functions. For instance, thecomputing device(s) can include one or more processor(s) and one or moretangible, non-transitory, computer readable media. The one or moretangible, non-transitory, computer readable media can store instructionsthat when executed by the one or more processor(s) cause the vehicle 103(e.g., its computing system, one or more processors, etc.) to performoperations and functions, such as those described herein.

As shown in FIG. 1, the vehicle 103 can include one or more sensors 108,an autonomy computing system 110, vehicle control system 112,human-machine interface system 134, mode manager 136, and collisionmitigation system 138. One or more of these systems can be configured tocommunicate with one another via a communication channel. Thecommunication channel can include one or more data buses (e.g.,controller area network (CAN)), on-board diagnostics connector (e.g.,OBD-II), and/or a combination of wired and/or wireless communicationlinks. The on-board systems can send and/or receive data, messages,signals, etc. amongst one another via the communication channel.

The sensor(s) 108 can be configured to acquire sensor data 114associated with one or more objects that are proximate to the vehicle103 (e.g., within a field of view of one or more of the sensor(s) 108).The sensor(s) 108 can include a Light Detection and Ranging (LIDAR)system, a Radio Detection and Ranging (RADAR) system, one or morecameras (e.g., visible spectrum cameras, infrared cameras, etc.), motionsensors, and/or other types of imaging capture devices and/or sensors.The sensor data 114 can include image data, radar data, LIDAR data,and/or other data acquired by the sensor(s) 108. The object(s) caninclude, for example, pedestrians, vehicles, bicycles, and/or otherobjects. The object(s) can be located in front of, to the rear of,and/or to the side of the vehicle 103. The sensor data 114 can beindicative of locations associated with the object(s) within thesurrounding environment of the vehicle 103 at one or more times. Thesensor(s) 108 can provide the sensor data 114 to the autonomy computingsystem 110.

As shown in FIG. 2, the autonomy computing system 110 can retrieve orotherwise obtain map data 116, in addition to the sensor data 114. Themap data 116 can provide detailed information about the surroundingenvironment of the vehicle 103. For example, the map data 116 canprovide information regarding: the identity and location of differentroadways, road segments, buildings, or other items or objects (e.g.,lampposts, crosswalks, curbing, etc.); the location and directions oftraffic lanes (e.g., the location and direction of a parking lane, aturning lane, a bicycle lane, or other lanes within a particular roadwayor other travel way and/or one or more boundary markings associatedtherewith); traffic control data (e.g., the location and instructions ofsignage, traffic lights, or other traffic control devices); and/or anyother map data that provides information that assists the vehicle 103 incomprehending and perceiving its surrounding environment and itsrelationship thereto.

The autonomy computing system 110 can include a perception system 120, aprediction system 122, a motion planning system 124, and/or othersystems that cooperate to perceive the surrounding environment of thevehicle 103 and determine a motion plan for controlling the motion ofthe vehicle 103 accordingly. For example, the autonomy computing system110 can receive the sensor data 114 from the sensor(s) 108, attempt tocomprehend the surrounding environment by performing various processingtechniques on the sensor data 114 (and/or other data), and generate anappropriate motion plan through such surrounding environment. Theautonomy computing system 110 can control the one or more vehiclecontrol systems 112 to operate the vehicle 103 according to the motionplan.

The autonomy computing system 110 can identify one or more objects thatare proximate to the vehicle 103 based at least in part on the sensordata 114 and/or the map data 116. For example, the perception system 120can obtain perception data 126 descriptive of a current state of anobject that is proximate to the vehicle 103. The perception data 126 foreach object can describe, for example, an estimate of the object's:current location (also referred to as position); current speed (alsoreferred to as velocity); current acceleration; current heading; currentorientation; size/footprint (e.g., as represented by a boundingpolygon); class (e.g., pedestrian class vs. vehicle class vs. bicycleclass), and/or other state information. In some implementations, theperception system 120 can determine perception data 126 for each objectover a number of iterations. In particular, the perception system 120can update the perception data 126 for each object at each iteration.Thus, the perception system 120 can detect and track objects (e.g.,vehicles, pedestrians, bicycles, and the like) that are proximate to theautonomous vehicle 103 over time. The perception system 120 can providethe perception data 126 to the prediction system 122 (e.g., forpredicting the movement of an object).

The prediction system 122 can create predicted data 128 associated witheach of the respective one or more objects proximate to the vehicle 103.The predicted data 128 can be indicative of one or more predicted futurelocations of each respective object. The predicted data 128 can beindicative of a predicted path (e.g., predicted trajectory) of at leastone object within the surrounding environment of the vehicle 103. Forexample, the predicted path (e.g., trajectory) can indicate a path alongwhich the respective object is predicted to travel over time (and/or thespeed at which the object is predicted to travel along the predictedpath). The prediction system 122 can provide the predicted data 128associated with the object(s) to the motion planning system 124.

The motion planning system 124 can determine a motion plan for thevehicle 103 based at least in part on the predicted data 128 (and/orother data), and save the motion plan as motion plan data 130. Themotion plan data 130 can include vehicle actions with respect to theobjects proximate to the vehicle 103 as well as the predicted movements.For instance, the motion planning system 124 can implement anoptimization algorithm that considers cost data associated with avehicle action as well as other objective functions (e.g., based onspeed limits, traffic lights, etc.), if any, to determine optimizedvariables that make up the motion plan data 130. By way of example, themotion planning system 124 can determine that the vehicle 103 canperform a certain action (e.g., pass an object) without increasing thepotential risk to the vehicle 103 and/or violating any traffic laws(e.g., speed limits, lane boundaries, signage). The motion plan data 130can include a planned trajectory, speed, acceleration, etc. of thevehicle 103.

The motion planning system 124 can provide at least a portion of themotion plan data 130 that indicates one or more vehicle actions, aplanned trajectory, and/or other operating parameters to the vehiclecontrol system(s) 112 to implement the motion plan for the vehicle 103.For instance, the vehicle 103 can include a mobility controllerconfigured to translate the motion plan data 130 into instructions. Byway of example, the mobility controller can translate the motion plandata 130 into instructions to adjust the steering of the vehicle 103 “X”degrees, apply a certain magnitude of braking force, etc. The mobilitycontroller can send one or more control signals to the responsiblevehicle control sub-system (e.g., braking control system 214, steeringcontrol system 218, acceleration control system 216) to execute theinstructions and implement the motion plan.

The vehicle 103 can include a communications system 132 configured toallow the vehicle computing system 102 (and its computing device(s)) tocommunicate with other computing devices. The vehicle computing system102 can use the communications system 132 (e.g., shown in FIG. 1) tocommunicate with the operations computing system 104 and/or one or moreother remote computing device(s) over one or more networks (e.g., viaone or more wireless signal connections). In some implementations, thecommunications system 132 can allow communication among one or more ofthe system(s) on-board the vehicle 103. The communications system 132can include any suitable components for interfacing with one or morenetwork(s), including, for example, transmitters, receivers, ports,controllers, antennas, and/or other suitable components that can helpfacilitate communication.

The vehicle 103 can include a human-machine interface system 134 (e.g.,as shown in FIG. 2) configured to control one or more indicator(s). Thehuman-machine interface system 134 can control one or more of visualindicator(s) 236 (e.g., light(s), display(s), etc.), tactileindicator(s) 238 (e.g., ultrasound emitter(s), vibration motor(s),etc.), and audible indicator(s) 240 (e.g., speaker(s), etc.) locatedon-board the vehicle 103. The human-machine interface system 134 canreceive control signals from one or more system(s) on-board the vehicle103 to control the one or more indicator(s). For example, if atrajectory of the vehicle 103 includes an upcoming left turn, then theautonomy computing system 110 can control the human-machine interfacesystem 134 (e.g., via one or more control signal(s)) to activate a leftturn signal light of the vehicle 103. In another example, a human drivercan control the human-machine interface system 134 (e.g., via a turnsignal lever) to activate a left turn signal light of the vehicle 103.

The vehicle 103 can include a mode manager 136 (e.g., as shown in FIG.2) configured to control one or more sub-system(s) for managing anoperating mode of the vehicle 103. For instance, the mode manager 136can include a prerequisite system 146 and a mode-setting system 148. Themode manager 136 can receive a requested operating mode from one or moresystem(s) on-board the vehicle 103 (e.g., from the autonomy computingsystem 110, human-machine interface system 134, etc.). For example, ifthe autonomy computing system 110 encounters an error, the autonomycomputing system 110 can send one or more control signal(s) to the modemanager 136 to switch an operating mode of the vehicle 103 from a fullyautonomous operating mode 106A to a manual operating mode 106C. Asanother example, an operator (e.g., human driver) can control thehuman-machine interface system 134 to send one or more control signal(s)to the mode manager 136 to switch an operating mode of the vehicle 103from a manual operating mode 106C to a fully autonomous operating mode106A.

The mode manager 136 can compare the requested operating mode with acurrent operating mode of the vehicle 103, and if the requestedoperating mode is different from the current operating mode, then themode manager 136 can determine one or more prerequisite(s) to switchingthe vehicle 103 to the requested operating mode. The mode manager 136can determine whether the prerequisite(s) are met, and if so the modemanager 136 can provide one or more control signal(s) to one or moresystem(s) on-board the vehicle 103 to switch an operating mode of thevehicle 103 to the requested operating mode. By way of example, if avehicle 103 is operating in fully autonomous operating mode 106A andreceives a request to switch to manual operating mode 106C, then themode manager 136 can determine that a prerequisite to switching theoperating mode is an availability of a driver. The mode manager 136 candetermine if this prerequisite is met by controlling the vehicle 103 toprompt an operator of the vehicle 103 to affirm that operator will takecontrol of the vehicle 103. The mode manager 136 can determine yetanother prerequisite to switching the operating mode is that the vehicle103 is not currently executing a turning maneuver. The mode manager 136can communicate with the vehicle control system 112 or other system(s)on-board the vehicle 103 to determine if this prerequisite is met. Ifthe vehicle 103 is executing a turning maneuver, the mode manager 136can wait for the turning maneuver to be completed, or the mode manager136 can determine that the prerequisite is not met. As another example,the operations computing system 104 can send a communication to thevehicle 103 instructing the vehicle 103 to enter into the autonomousoperating mode 106A when providing a rideshare service to a user. Themode manager 136 can determine one or more prerequisite(s) to enteringthe fully autonomous operating mode 106A, determine if theprerequisite(s) are met, and switch the vehicle to the fully autonomousoperating mode 106A.

The vehicle 103 can include a collision mitigation system 138 (e.g., asshown in FIG. 2) configured to control one or more sub-system(s) fordetecting an avoiding a potential collision. For instance, the collisionmitigation system 138 can include a configurable data structure 140,collision detection system 142, and a collision avoidance system 144.The collision mitigation system 138 can monitor a surroundingenvironment of the vehicle 103 using one or more sensors (e.g., a RadioDetection and Ranging (RADAR) system, one or more cameras, and/or othertypes of image capture devices and/or sensors) to detect a potentialcollision between the vehicle 103 and an object in the surroundingenvironment. When a potential collision is detected, the collisionmitigation system 138 can control the vehicle 103 to avoid the potentialcollision, based at least in part on an operating mode of the vehicle103. For example, if vehicle 103 is operating in fully autonomousoperating mode 106A and the collision mitigation system 138 detects apotential collision, the collision mitigation system 138 can provideinformation on the potential collision to the autonomy computing system110 that can adjust a trajectory if the vehicle 103 to avoid thepotential collision. For instance, to adjust the trajectory of thevehicle 103 such information can be provided to the motion planningsystem 124, which can consider the cost (e.g., very high cost,overriding/superseding cost, etc.) of colliding with another object whenplanning the motion of the vehicle 103. As such, the motion planningsystem 124 can create a motion plan 130 by which the vehicle 103 followsa trajectory to avoid the collision. As another example, if vehicle 103is operating in manual mode 106C and the collision mitigation system 136detects a potential collision, the collision mitigation system 138 cancontrol the human-machine interface system 134 to display a warninglight to operator(s) of the vehicle 103.

In some implementations, the collision mitigation system 138 can controla motion of the vehicle 103 to avoid a potential collision. For example,if a potential collision persists for a time after the collisionmitigation system 138 provides information on the potential collision tothe autonomy computing system 110, then the collision mitigation system138 can provide one or more control signal(s) to control the vehicle 103to execute a braking maneuver. As another example, if a potentialcollision persists for a time after the collision mitigation system 138controls the vehicle 103 to display a warning light, then the collisionmitigation system 138 can provide one or more control signal(s) tocontrol the vehicle 103 to execute a braking maneuver.

As shown in FIG. 3, the configurable data structure 140 can store one ormore operating mode(s) 302, one or more response characteristic(s) 303,and one or more corresponding value(s) 304. The configurable datastructure 300 can store the operating mode(s) 302 in relation to anindex 301. For example, the operating mode(s) 302 can include passenger1 mode 321, driver 1 mode 324, and default service mode 327. Each of theoperating mode(s) 302 is associated with one or more responsecharacteristic(s) 303. For example, the passenger 1 mode 321 isassociated with response characteristic(s) 331, 332, 333, and 334; thedriver 1 mode 324 is associated with response characteristic(s) 331,332, 335, and 336; and the default service mode 327 is associated withresponse characteristic(s) 332, 337, and 338. Each of the responsecharacteristic(s) 303 is associated with corresponding value(s) 304. Forexample, response characteristic(s) 331, 332, 333, and 334 (e.g., forthe passenger 1 mode) are associated with corresponding value(s) 3401,3402, 3403, and 3404, respectively; response characteristic(s) 331, 332,335, and 336 (e.g., for the driver 1 mode) are associated withcorresponding value(s) 3405, 3406, 3407, and 3408, respectively; andresponse characteristic(s) 332, 337, and 338 (e.g., for the defaultservice mode) are associated with corresponding value(s) 3409, 3410, and3411, respectively. A response characteristic of the vehicle 103 can beassociated with zero or more of the operating mode(s) 302. For example,response characteristic(s) 331 and 332 are associated with operatingmode 321 and operating mode 324. As another example, the vehicle 103 caninclude one or more response characteristic(s) in addition to responsecharacteristic(s) 331-338 that are not associated with any one of theoperating mode(s) 302. Additionally, a response characteristic of thevehicle 103 can be associated with a range of values for eachcorresponding value. For example, response characteristic 331 representswhether the collision mitigation system 138 should control thehuman-machine interface system 134 to flash a warning light. Theresponse characteristic 331 is associated with the range of values“yes/no.” Accordingly, the corresponding values 3401 and 3405 can beeither a “yes” or “no.” Furthermore, each of the corresponding value(s)304 is independent of one another. For example, the corresponding value3401 can be “yes” and the corresponding value 3405 can be “no.” Asanother example, response characteristic 334 represents a minimumdistance to keep with other vehicles and is associated with the range of10 feet, 15 feet, or 20 feet. As yet another example, responsecharacteristic 335 represents a maximum acceleration for the vehicle 103and is associated with the range between 1.0 and 4.0 m/s².

FIGS. 4A and 4B depict diagrams 402 and 404 illustrate a response of thevehicle 103 when the collision mitigation system 138 detects a potentialcollision. The response of the vehicle 103 can differ based at least inpart on an operating mode of the vehicle 103. For example, FIG. 4A showsa response of the vehicle 103 when operating in fully autonomous mode106A and FIG. 4B shows a response of the vehicle 103 when operating inmanual mode 106C.

As shown in FIG. 4A, at time t=0 the collision mitigation system 138detects a potential collision while the vehicle 103 is operating infully autonomous mode 106A. Immediately upon detecting the potentialcollision, at t=0 the collision mitigation system 138 controls thevehicle 103 to by providing information on the potential collision tothe autonomy computing system 110. The collision mitigation system 138then waits for the autonomy computing system 110 to adjust a trajectoryof the vehicle 103 to avoid the potential collision. At t=1, thecollision mitigation system 138 checks if the potential collisionpersists. If the potential collision persists, the collision mitigationsystem 138 can continue to wait for the autonomy computing system 110.At t=2, the collision mitigation system 138 checks if the potentialcollision still persists. If the potential collision persists, thecollision mitigation system 138 can control a motion of the vehicle 103to avoid the potential collision by providing control signal(s) to thevehicle control system 112 to throttle the vehicle 103 (e.g., todecelerate the vehicle 103). At t=3, the collision mitigation system 138checks if the potential collision still persists. If the potentialcollision persists, the collision mitigation system 138 provides controlsignal(s) to the vehicle control system 112 to execute a partial brakemaneuver. At t=4, the collision mitigation system 138 checks if thepotential collision still persists. If the potential collision persists,the collision mitigation system 138 can provide control signal(s) to thevehicle control system 112 to execute a full brake maneuver.

In some implementations, the duration between throttling at t=2 andexecuting a full brake at t=4 can be adjusted based at least in part ona response characteristic associated with the autonomous operating mode.For example, the vehicle 103 can include the response characteristic“maximum deceleration rate” and the fully autonomous mode 106A of thevehicle 103 can be associated with this response characteristic. Thecollision mitigation system 138 can expand the time between throttlingand executing a full brake if the corresponding value for “maximumdeceleration rate” is low, or contract the time between throttling andexecuting a full brake if the corresponding value for “maximumdeceleration rate” is high. Alternatively, the collision mitigationsystem 138 can control the motion of the vehicle 103 such that adeceleration of the vehicle 103 is less than the corresponding value for“maximum deceleration rate.”

As shown in FIG. 4B, at time t=0 the collision mitigation system 138detects a potential collision while the vehicle 103 is operating inmanual mode 106C. Upon detecting the potential collision, the collisionmitigation system 138 waits for a driver to control the vehicle 103 toavoid the potential collision. At t=1, the collision mitigation system138 checks if the potential collision persists. If the potentialcollision persists, the collision mitigation system 138 can control thehuman-machine interface system 134 to display a warning light in casethe driver has not noticed the potential collision. At t=2, thecollision mitigation system 138 checks if the potential collision stillpersists. If the potential collision persists, the collision mitigationsystem 138 can continue waiting for the driver to control the vehicle103 to avoid the potential collision. At t=3, the collision mitigationsystem 138 checks if the potential collision still persists. If thepotential collision persists, the collision mitigation system 138 cancontinue waiting for the driver to control the vehicle 103 to avoid thepotential collision. At t=4, the collision mitigation system 138 checksif the potential collision still persists. If the potential collisionpersists, the collision mitigation system 138 can provide controlsignal(s) to the vehicle control system 112 to execute a full brakemaneuver.

In some implementations, the duration between notifying the driver att=2 and executing a full brake at t=4 can be adjusted based at least inpart on a response characteristic associated with the manual operatingmode 106C. For example, the vehicle 103 can include the responsecharacteristic “maximum deceleration rate” and the manual operating mode160C of the vehicle 103 can be associated with this responsecharacteristic. The collision mitigation system 138 can expand the timebetween throttling and executing a full brake if the corresponding valuefor “maximum deceleration rate” is high, or contract the time betweenthrottling and executing a full brake if the corresponding value for“maximum deceleration rate” is low. Alternatively, the collisionmitigation system 138 can control the motion of the vehicle 103 suchthat a deceleration of the vehicle 103 is less than the correspondingvalue for “maximum deceleration rate.”

FIG. 5 depicts a flow diagram of an example method 500 of controllingthe vehicle 103 based at least in part on an operating mode of thevehicle 103 according to example embodiments of the present disclosure.One or more portion(s) of the method 500 can be implemented by one ormore computing device(s) such as, for example, the computing device(s)601 shown in FIG. 6. Moreover, one or more portion(s) of the method 500can be implemented as an algorithm on the hardware components of thedevice(s) described herein (e.g., as in FIGS. 1 and 6) to, for example,switch an operating mode of the vehicle 103. FIG. 5 depicts elementsperformed in a particular order for purposes of illustration anddiscussion. Those of ordinary skill in the art, using the disclosuresprovided herein, will understand that the elements of any of the methods(e.g., of FIG. 5) discussed herein can be adapted, rearranged, expanded,omitted, combined, and/or modified in various ways without deviatingfrom the scope of the present disclosure.

At (501), the method 500 can include receiving an operating mode. Forexample, the collision mitigation system 138 can receive data indicativeof the operating mode from the mode manager 136 when the mode manager136 sets the operating mode.

At (502), the method 500 can include determining responsecharacteristic(s). For example, the collision mitigation system 138 cansearch the configurable data structure 140 for an operation mode 302that matches the received operating mode. If a match is found, thecollision mitigation system 138 can determine the responsecharacteristic(s) 303 associated with the operating mode.

At (503), the method 500 can include detecting a potential collisionbetween the vehicle 103 and an object in the surrounding environment.For example, the collision mitigation system 138 can monitor thesurrounding environment of the vehicle 103 using one or more sensors(e.g., a Radio Detection and Ranging (RADAR) system, one or morecameras, and/or other types of image capture devices and/or sensors) todetect the potential collision.

At (504), the method 500 can include controlling the vehicle 103 toavoid the potential collision. For example, if the operating mode of thevehicle 103 is default autonomous mode 320, then one or more responsecharacteristic(s) 303 and corresponding value(s) 304 associated withdefault autonomous mode 320 can indicate that the collision mitigationsystem 138 is to immediately provide information on the potentialcollision to the autonomy computing system 110. As another example, ifthe operating mode of the vehicle 103 is driver 1 mode 324, then one ormore response characteristic(s) 303 and corresponding value(s) 304associated with driver 1 mode 324 can indicate that the collisionmitigation system 138 is to control the human-machine interface system134 to indicate the potential collision after a first duration of time,and to control the vehicle control system 112 to execute a full brakemaneuver after a second duration of time.

FIG. 6 depicts an example computing system 600 according to exampleembodiments of the present disclosure. The example system 600illustrated in FIG. 6 is provided as an example only. The components,systems, connections, and/or other aspects illustrated in FIG. 6 areoptional and are provided as examples of what is possible, but notrequired, to implement the present disclosure. The example system 600can include the vehicle computing system 102 of the vehicle 103 and, insome implementations, a remote computing system 610 including one ormore remote computing device(s) that are remote from the vehicle 103(e.g., the operations computing system 104) that can be communicativelycoupled to one another over one or more networks 620. The remotecomputing system 610 can be associated with a central operations systemand/or an entity associated with the vehicle 103 such as, for example, avehicle owner, vehicle manager, fleet operator, service provider, etc.

The computing device(s) 601 of the vehicle computing system 102 caninclude processor(s) 602 and a memory 604. The one or more processors602 can be any suitable processing device (e.g., a processor core, amicroprocessor, an ASIC, a FPGA, a controller, a microcontroller, etc.)and can be one processor or a plurality of processors that areoperatively connected. The memory 604 can include one or morenon-transitory computer-readable storage media, such as RAM, ROM,EEPROM, EPROM, one or more memory devices, flash memory devices, etc.,and combinations thereof.

The memory 604 can store information that can be accessed by the one ormore processors 602. For instance, the memory 604 (e.g., one or morenon-transitory computer-readable storage mediums, memory devices)on-board the vehicle 103 can include computer-readable instructions 606that can be executed by the one or more processors 602. The instructions606 can be software written in any suitable programming language or canbe implemented in hardware. Additionally, or alternatively, theinstructions 606 can be executed in logically and/or virtually separatethreads on processor(s) 602.

For example, the memory 604 on-board the vehicle 103 can storeinstructions 606 that when executed by the one or more processors 602on-board the vehicle 103 cause the one or more processors 602 (thevehicle computing system 102) to perform operations such as any of theoperations and functions of the vehicle computing system 102, asdescribed herein, the operations for controlling the vehicle 103 basedat least in part on an operating mode of the vehicle 103.

The memory 604 can store data 608 that can be obtained, received,accessed, written, manipulated, created, and/or stored. The data 608 caninclude, for instance, data associated with an operating mode of thevehicle, data associated with response characteristics, data structures(as described herein), sensor data, perception data, prediction data,motion planning data, and/or other data/information as described herein.In some implementations, the computing device(s) 601 can obtain datafrom one or more memory device(s) that are remote from the vehicle 103.

The computing device(s) 601 can also include a communication interface609 used to communicate with one or more other system(s) on-board thevehicle 103 and/or a remote computing device that is remote from thevehicle 103 (e.g., of remote computing system 610). The communicationinterface 609 can include any circuits, components, software, etc. forcommunicating via one or more networks (e.g., 620). In someimplementations, the communication interface 609 can include, forexample, one or more of a communications controller, receiver,transceiver, transmitter, port, conductors, software, and/or hardwarefor communicating data.

The network(s) 620 can be any type of network or combination of networksthat allows for communication between devices. In some embodiments, thenetwork(s) can include one or more of a local area network, wide areanetwork, the Internet, secure network, cellular network, mesh network,peer-to-peer communication link, and/or some combination thereof, andcan include any number of wired or wireless links. Communication overthe network(s) 620 can be accomplished, for instance, via acommunication interface using any type of protocol, protection scheme,encoding, format, packaging, etc.

The remote computing system 610 can include one or more remote computingdevices that are remote from the vehicle computing system 102. Theremote computing devices can include components (e.g., processor(s),memory, instructions, data) similar to that described herein for thecomputing device(s) 601. Moreover, the remote computing system 610 canbe configured to perform one or more operations of the operationscomputing system 104, as described herein.

Computing tasks discussed herein as being performed at computingdevice(s) remote from the vehicle can instead be performed at thevehicle (e.g., via the vehicle computing system), or vice versa. Suchconfigurations can be implemented without deviating from the scope ofthe present disclosure. The use of computer-based systems allows for agreat variety of possible configurations, combinations, and divisions oftasks and functionality between and among components.Computer-implemented operations can be performed on a single componentor across multiple components. Computer-implemented tasks and/oroperations can be performed sequentially or in parallel. Data andinstructions can be stored in a single memory device or across multiplememory devices.

While the present subject matter has been described in detail withrespect to specific example embodiments and methods thereof, it will beappreciated that those skilled in the art, upon attaining anunderstanding of the foregoing can readily produce alterations to,variations of, and equivalents to such embodiments. Accordingly, thescope of the present disclosure is by way of example rather than by wayof limitation, and the subject disclosure does not preclude inclusion ofsuch modifications, variations and/or additions to the present subjectmatter as would be readily apparent to one of ordinary skill in the art.

1.-20. (canceled)
 21. A computer-implemented method for autonomousvehicle control, the method comprising: receiving, by a computing systemcomprising one or more computing devices, data indicative of anoperating mode of the autonomous vehicle, the autonomous vehicleconfigured to operate in a plurality of operating modes associated withan autonomy computing system of the autonomous vehicle; determining, bythe computing system, one or more response characteristics for reactingto a potential collision based at least in part on the operating mode ofthe autonomous vehicle and an operating mode of a collision mitigationsystem, each response characteristic indicative of a response byindicating how the collision mitigation system onboard the autonomousvehicle responds to the potential collision with an object in anenvironment proximate to the autonomous vehicle, wherein the collisionmitigation system is separate from the autonomy system of the autonomousvehicle; sending, by the computing system, one or more control signalsto one or more other systems onboard the autonomous vehicle based atleast in part on the one or more response characteristics; andcontrolling, by the computing system, the autonomous vehicle based atleast in part on the one or more control signals.
 22. Thecomputer-implemented method of claim 21, wherein the one or more controlsignals are indicative of a detection of the potential collision. 23.The computer-implemented method of claim 21, wherein: sending the one ormore control signals comprises: sending, by the computing system, one ormore first control signals to the autonomy computing system at a firsttime when the operating mode of the autonomous vehicle is an autonomousmode; and sending, by the computing system, one or more second controlsignals to the one or more other systems onboard the autonomous vehicle,in response to the one or more first control signals, the one or moresecond control signals indicating one or more vehicle maneuvers tocontrol the autonomous vehicle.
 24. The computer-implemented method ofclaim 21, wherein: sending the one or more control signals comprisessending, by the computing system, one or more third control signals to ahuman-machine interface system onboard the autonomous vehicle at asecond time when the operating mode of the autonomous vehicle is amanual driving mode; and wherein controlling the autonomous vehiclebased at least in part on the one or more control signals comprisescontrolling the autonomous vehicle based at least in part on an inputfrom an operator of the autonomous vehicle subsequent to sending the oneor more third control signals.
 25. The computer-implemented method ofclaim 21, wherein the one or more control signals indicate one or morevehicle maneuvers to avoid the potential collision; and whereincontrolling the autonomous vehicle based at least in part on the one ormore control signals comprises controlling the autonomous vehicle toavoid the potential collision based at least in part on the one or morecontrol signals.
 26. The computer-implemented method of claim 25,further comprising: monitoring, by the computing system, the potentialcollision for a first duration of time; and sending, by the computingsystem, the one or more control signals to the one or more other systemswhen the potential collision is determined to persist after the firstduration of time.
 27. The computer-implemented method of claim 21,further comprising: detecting, by the computing system, the potentialcollision between the autonomous vehicle and the object in theenvironment proximate to the autonomous vehicle, wherein the potentialcollision is detected by the autonomy computing system independent ofthe detection by the collision mitigation system.
 28. Thecomputer-implemented method of claim 27, wherein detecting the potentialcollision between the autonomous vehicle and the object in theenvironment proximate to the autonomous vehicle comprises: obtaining, bythe computing system, trajectory information associated with theautonomous vehicle from the autonomy computing system; determining, bythe computing system, one or more regions of interest in the environmentproximate to the autonomous vehicle based at least in part on thetrajectory information; and detecting, by the computing system, thepotential collision within the one or more regions of interest whilemonitoring the one or more regions of interest for the potentialcollision.
 29. A computing system for autonomous vehicle control, thesystem comprising: one or more processors; and one or more computerreadable media that collectively store instructions that when executedby the one or more processors cause the computing system to performoperations, the operations comprising: receiving data indicative of anoperating mode of an autonomous vehicle, wherein the autonomous vehicleis configured to operate in a plurality of operating modes associatedwith an autonomy computing system of the autonomous vehicle; determiningone or more response characteristics for reacting to a potentialcollision based at least in part on the operating mode of the autonomousvehicle and an operating mode of a collision mitigation system, eachresponse characteristic indicative of a response by indicating how thecollision mitigation system onboard the autonomous vehicle responds tothe potential collision with an object in an environment proximate tothe autonomous vehicle, wherein the collision mitigation system isseparate from the autonomy system of the autonomous vehicle; sending oneor more control signals to one or more other systems onboard theautonomous vehicle based at least in part on the one or more responsecharacteristics; and controlling the autonomous vehicle based at leastin part on the one or more control signals.
 30. The computing system ofclaim 29, wherein: sending the one or more control signals comprises:sending one or more first control signals to the autonomy computingsystem at a first time when the operating mode of the autonomous vehicleis an autonomous mode; and sending one or more second control signals tothe one or more other systems onboard the autonomous vehicle, inresponse to the one or more first control signals, the one or moresecond control signals indicating one or more vehicle maneuvers tocontrol the autonomous vehicle.
 31. The computing system of claim 29,wherein: sending the one or more control signals comprises sending oneor more third control signals from the collision mitigation system to ahuman-machine interface system onboard the autonomous vehicle at asecond time when the operating mode of the autonomous vehicle is amanual driving mode; and wherein controlling the autonomous vehiclebased at least in part on the one or more control signals comprisescontrolling the autonomous vehicle based at least in part on an inputfrom an operator of the autonomous vehicle subsequent to sending the oneor more third control signals.
 32. The computing system of claim 29,wherein the one or more control signals indicate one or more vehiclemaneuvers to avoid the potential collision; and wherein controlling theautonomous vehicle based at least in part on the one or more controlsignals comprises controlling the autonomous vehicle to avoid thepotential collision based at least in part on the one or more controlsignals.
 33. The computing system of claim 30, wherein the operationsfurther comprise: monitoring, at the collision mitigation system, thepotential collision for a first duration of time; and sending thecontrol signals from the collision mitigation system to the one or moreother systems when the potential collision is determined to persistafter the first duration of time.
 34. The computing system of claim 29,wherein the operations further comprise: detecting the potentialcollision at the collision mitigation system, and detecting thepotential collision at the autonomy computing system independent of thedetection at the collision mitigation system.
 35. An autonomous vehicle,comprising: one or more processors; and one or more computer readablemedia that collectively store instructions that when executed by the oneor more processors cause the autonomous vehicle to perform operations,the operations comprising: receiving data indicative of an operatingmode of the autonomous vehicle, wherein the autonomous vehicle isconfigured to operate in a plurality of operating modes associated withan autonomy computing system of the autonomous vehicle; determining oneor more response characteristics for reacting to a potential collisionbased at least in part on the operating mode of the autonomous vehicleand an operating mode of a collision mitigation system, each responsecharacteristic indicative of a response by indicating how the collisionmitigation system onboard the autonomous vehicle responds to thepotential collision with an object in an environment proximate to theautonomous vehicle, wherein the collision mitigation system is separatefrom the autonomy system of the autonomous vehicle; sending one or morecontrol signals to one or more other systems onboard the autonomousvehicle based at least in part on the one or more responsecharacteristics; and controlling the autonomous vehicle based at leastin part on the one or more control signals.
 36. The autonomous vehicleof claim 35, wherein: sending the one or more control signals comprises:sending one or more first control signals from the collision mitigationsystem to the autonomy computing system at a first time when theoperating mode of the autonomous vehicle is an autonomous mode; andsending one or more second control signals from the autonomy computingsystem to the one or more other systems onboard the autonomous vehicle,in response to the one or more first control signals, the one or moresecond control signals indicating one or more vehicle maneuvers tocontrol the autonomous vehicle.
 37. The autonomous vehicle of claim 35,wherein: sending the one or more control signals comprises sending oneor more third control signals from the collision mitigation system to ahuman-machine interface system onboard the autonomous vehicle at asecond time when the operating mode of the autonomous vehicle is amanual driving mode; and wherein controlling the autonomous vehiclebased at least in part on the one or more control signals comprisescontrolling the autonomous vehicle based at least in part on an inputfrom an operator of the autonomous vehicle subsequent to sending the oneor more third control signals.
 38. The autonomous vehicle of claim 35,wherein the operations further comprise: sending one or more controlsignals from the collision mitigation system to the one or more othersystems onboard the autonomous vehicle based at least in part on the oneor more response characteristics, the one or more control signalsindicating one or more vehicle maneuvers to avoid the potentialcollision; and controlling the autonomous vehicle to avoid the potentialcollision based at least in part on the one or more control signals. 39.The autonomous vehicle of claim 38, wherein the operations furthercomprise: monitoring, at the collision mitigation system, the potentialcollision for a first duration of time; and sending the control signalsfrom the collision mitigation system to the one or more other systemswhen the potential collision is determined to persist after the firstduration of time.
 40. The autonomous vehicle of claim 35, wherein theoperations further comprise: detecting the potential collision at thecollision mitigation system; and detecting the potential collision atthe autonomy computing system independent of the detection at thecollision mitigation system.